Generic cleanup Metrics and Losses #203

JimClarke5 · 2021-02-01T20:01:12Z

This PR effects both losses and metrics.

This PR cleans up generic parameters to cut down on the number of unique parameters to at most 1 where possible.
For example:
public abstract class Metric<U extends TNumber, T extends TNumber> {
becomes
public abstract class Metric<T extends TNumber> {

and

@Override
  public <V extends TNumber> List<Op> updateStateList(Operand<U> values, Operand<V> sampleWeights) {

Changes to:

 public List<Op> updateStateList(
      Operand<? extends TNumber> values, Operand<? extends TNumber> sampleWeights)

Sync with master tensorflow on upstream

Merge main branch to local branch

Update after losses merge

Fix Javadoc errors (tensorflow#152)

pull type def

merge

Fix JavaDoc, modify one line code block to include braces.

…nly live within a single instance of a Metric.

…e same type as the return or internal variables,

…on when static shapes cannot boradcast.

change hasValidNonscalarShape to canBroadcastNonscalarShapes change hasValidNonscalarShape to canBroadcastNonscalarShapes

move the dynamic shapes and rank down to the dynamic section so they are created needlessly when static Fix if statement to check for unknown size and unknown dimensions

renamed WeightBroadcastTest to AssertBroadcastableTest and added BroadcastWeightsTest

…e same type as the return or internal variables,

…on when static shapes cannot boradcast.

change hasValidNonscalarShape to canBroadcastNonscalarShapes change hasValidNonscalarShape to canBroadcastNonscalarShapes

move the dynamic shapes and rank down to the dynamic section so they are created needlessly when static Fix if statement to check for unknown size and unknown dimensions

renamed WeightBroadcastTest to AssertBroadcastableTest and added BroadcastWeightsTest

* Successfully remove extra type params, but it broke javadoc generation Signed-off-by: Ryan Nett <[email protected]> * Generate covariant types Signed-off-by: Ryan Nett <[email protected]> * Do generation Signed-off-by: Ryan Nett <[email protected]> * Update help text. Signed-off-by: Ryan Nett <[email protected]> * Fixes Signed-off-by: Ryan Nett <[email protected]>

….sparse.sparseToDense with the output of tf.sparse.denseToDenseSetOperation

google-cla · 2021-02-01T20:01:50Z

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

JimClarke5 · 2021-02-01T20:07:50Z

@karllessard @Craigacp @rnett Could one of you clue me in what went wrong here. I rebased the project to the latest master, but it still seem to pick up a lot of the commits from the metrics1 branch, that were applied with yesterday's merge.

rnett · 2021-02-01T21:41:36Z

Only time I've seen that happen is when I didn't update my local master, or I had a commit in my branch (that wasn't in master) from before master's latest commit.

Can you cherry pick the commits you want to a new branch? That's how I've handled it.

JimClarke5 · 2021-02-01T21:51:39Z

I found this article which sort of explains how to do it. Seems overly complicated.
How to Fix Your Git Branches After a Rebase

Craigacp · 2021-02-01T21:56:46Z

I second Ryan's suggestion of git cherry-pick. Make a new branch from master, run cherry pick giving it the commits that should be in this PR, then force push that branch over this one (or make a new pr from the new branch).

There are other ways to fix it, but it depends on what the exact state of your git repo is.

JimClarke5 · 2021-02-01T21:58:18Z

I had master->metrics1->generic_cleanup

I pulled yesterday's merge to my master from the main TF repository.
Then did a rebase of generic_cleanup to my master.

Based on recreating generic_cleanup with cherry-pick,
should I fix this PR? I assume cancel/replace.

Craigacp · 2021-02-01T22:18:05Z

A force push will fix it, but it'll require a bit of a renaming dance with the branches (i.e. cherry-pick to new branch, delete old branch, rename new branch to old branch, force push). It's up to you whichever you find easiest.

JimClarke5 · 2021-02-01T23:32:59Z

Rebuilt the branch to exclude commits already included in master.

JimClarke5 added 30 commits October 8, 2020 13:19

Merge pull request #3 from tensorflow/master

c57a2e7

Sync with master tensorflow on upstream

Merge pull request #4 from tensorflow/master

09fc07e

Merge main branch to local branch

Merge pull request #5 from tensorflow/master

a99dcb4

Update after losses merge

Merge pull request #6 from tensorflow/master

ba294ea

Fix Javadoc errors (tensorflow#152)

Merge pull request #7 from tensorflow/master

04f419a

pull type def

Merge pull request #8 from tensorflow/master

02e7ebf

merge

Initial checkin

8f57a7a

Initial checkin and sync with master

090bde4

Initial checkin and sync with master

ce5fa27

JavaDoc cleanup

475ca36

Javadoc fixes

b48c09a

Change LossInterface to LossMetric.

c4c06de

Fix JavaDoc, modify one line code block to include braces.

Removed hashmap for variables, they are not needed as the variables o…

2d4c17b

…nly live within a single instance of a Metric.

reformat code

fbb12f4

Add tests for assertBroadcastable

30196af

Change type to resultType

1248581

Added V data type for sampleWeights so that it is not forced to be th…

96bd55b

…e same type as the return or internal variables,

change 'type' to 'resultType'

d706f55

clean up mean and fix assert assertBroadcastable

0147442

fix error message

c0c127f

Change sampleWeights to have its own generic type <S extends TNumber>

89ec9ed

Add commment about invalid tests expecting IllegalArgumentExceptions

1b81c82

Add this exception instead of the more generic IllegalArgumentExcepti…

54d4ae9

…on when static shapes cannot boradcast.

change IllegalArgumentException to NotBroadcastableException.

ca50dfa

change hasValidNonscalarShape to canBroadcastNonscalarShapes change hasValidNonscalarShape to canBroadcastNonscalarShapes

reformat code

ca5ad09

Fis=x Javadoc

1e72017

move the dynamic shapes and rank down to the dynamic section so they are created needlessly when static Fix if statement to check for unknown size and unknown dimensions

Fix Reduce to use boradcastWeights,

85bdde7

renamed WeightBroadcastTest to AssertBroadcastableTest and added BroadcastWeightsTest

Added comment to count to indicate that it may be weighted.

3f46bd2

Added SetsOps and fixed AssertBroadcastable to use SetsOps methods,

fe65ae7

Fixed based on various PR comments.

f8d38cf

JimClarke5 and others added 19 commits February 1, 2021 13:14

Added V data type for sampleWeights so that it is not forced to be th…

d475b1a

…e same type as the return or internal variables,

change 'type' to 'resultType'

8f530cc

clean up mean and fix assert assertBroadcastable

947482a

fix error message

f473568

Change sampleWeights to have its own generic type <S extends TNumber>

99ff15d

Add commment about invalid tests expecting IllegalArgumentExceptions

f4e3e04

Add this exception instead of the more generic IllegalArgumentExcepti…

4efdb62

…on when static shapes cannot boradcast.

change IllegalArgumentException to NotBroadcastableException.

b0a143e

change hasValidNonscalarShape to canBroadcastNonscalarShapes change hasValidNonscalarShape to canBroadcastNonscalarShapes

reformat code

5e05018

Fis=x Javadoc

b7530a3

move the dynamic shapes and rank down to the dynamic section so they are created needlessly when static Fix if statement to check for unknown size and unknown dimensions

Fix Reduce to use boradcastWeights,

a2761f0

renamed WeightBroadcastTest to AssertBroadcastableTest and added BroadcastWeightsTest

Added comment to count to indicate that it may be weighted.

c2efa2a

Added SetsOps and fixed AssertBroadcastable to use SetsOps methods,

9db5767

Fixed based on various PR comments.

fb8aa65

Deleted, no longer needed after change to Variable handling in Metrics.

02be174

Fix SetOps to properly convert sparse tensor to dense tensor using tf…

a1c5187

….sparse.sparseToDense with the output of tf.sparse.denseToDenseSetOperation

Simplify generic parameters across losses and metrics.

b125294

Reformat code

78366de

JimClarke5 closed this Feb 1, 2021

JimClarke5 deleted the generic_cleanup branch February 1, 2021 23:42

JimClarke5 mentioned this pull request Feb 1, 2021

Generic cleanup Metrics and Losses #204

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Generic cleanup Metrics and Losses #203

Generic cleanup Metrics and Losses #203

Uh oh!

JimClarke5 commented Feb 1, 2021

Uh oh!

google-cla bot commented Feb 1, 2021

Uh oh!

JimClarke5 commented Feb 1, 2021

Uh oh!

rnett commented Feb 1, 2021 •

edited

Loading

Uh oh!

JimClarke5 commented Feb 1, 2021

Uh oh!

Craigacp commented Feb 1, 2021

Uh oh!

JimClarke5 commented Feb 1, 2021 •

edited

Loading

Uh oh!

Craigacp commented Feb 1, 2021

Uh oh!

JimClarke5 commented Feb 1, 2021

Uh oh!

Uh oh!

Generic cleanup Metrics and Losses #203

Generic cleanup Metrics and Losses #203

Uh oh!

Conversation

JimClarke5 commented Feb 1, 2021

Uh oh!

google-cla bot commented Feb 1, 2021

Uh oh!

JimClarke5 commented Feb 1, 2021

Uh oh!

rnett commented Feb 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JimClarke5 commented Feb 1, 2021

Uh oh!

Craigacp commented Feb 1, 2021

Uh oh!

JimClarke5 commented Feb 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Craigacp commented Feb 1, 2021

Uh oh!

JimClarke5 commented Feb 1, 2021

Uh oh!

Uh oh!

rnett commented Feb 1, 2021 •

edited

Loading

JimClarke5 commented Feb 1, 2021 •

edited

Loading